Point cloud data has sparsity, irregularity, and permutation invariance, and lacks topological information, which makes it difficult to extract features of point cloud. Therefore, a Siamese Adaptive Graph Convolution Algorithm (SAGCA) was proposed for point cloud classification and segmentation. Firstly, the topological relationships between irregular and sparse point cloud features were mined by constructing feature relationship graph. Then, the Siamese composition idea of sharing convolution learning weights was introduced to ensure the permutation invariance of point cloud data and make the topological relationship expression more accurate. Finally, SAGCA was combined with various deep learning networks for processing point cloud data by both global and local combination methods, thereby enhancing the feature extraction ability of the network. Comparison results with PointNet++ benchmark network of the classification, object part segmentation and scene semantic segmentation experiments on ScanObjectNN, ShapeNetPart and S3DIS datasets, respectively, show that, based on the same dataset and evaluation criteria, SAGCA has the class mean Accuracy (mAcc) of classification increased by 2.80 percentage points, the overall class average Intersection over Union (IoU) of part segmentation increased by 2.31 percentage points, and the class mean Intersection over Union (mIoU) of scene semantic segmentation increased by 2.40 percentage points, verifying that SAGCA can effectively enhance the feature extraction ability of the network and is suitable for multiple point cloud classification and segmentation tasks.